Understanding autoencoders with information theoretic concepts
نویسندگان
چکیده
منابع مشابه
Understanding Autoencoders with Information Theoretic Concepts
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks. In this paper, we illustrate an advanced information theoretic methodology to understand the dynamics of learning and the design of autoencoders, a special type of deep learning architectures that resembles a communication channel. By generalizing t...
متن کاملApplication of Information - Theoretic Concepts in Chemoinformatics
The use of computational methodologies for chemical database mining and molecular similarity searching or structure-activity relationship analysis has become an integral part of modern chemical and pharmaceutical research. These types of computational studies fall into the chemoinformatics spectrum and usually have large-scale character. Concepts from information theory such as Shannon entropy ...
متن کاملConditional Autoencoders with Adversarial Information Factorization
Generative models, such as variational auto-encoders (VAE) and generative adversarial networks (GAN), have been immensely successful in approximating image statistics in computer vision. VAEs are useful for unsupervised feature learning, while GANs alleviate supervision by penalizing inaccurate samples using an adversarial game. In order to utilize benefits of these two approaches, we combine t...
متن کاملInformation-Theoretic Concepts for the Analysis of Complex Networks
& In this article, we present information-theoretic concepts for analyzing complex networks. We see that the application of information-theoretic concepts to networks leads to interesting tasks and gives a possibility for understanding information processing in networks. The main contribution of this article is a method for determining the structural information content of graphs that is based ...
متن کاملInfoVAE: Information Maximizing Variational Autoencoders
It has been previously observed that variational autoencoders tend to ignore the latent code when combined with a decoding distribution that is too flexible. This undermines the purpose of unsupervised representation learning. In this paper, we additionally show that existing training criteria can lead to extremely poor amortized inference distributions and overestimation of the posterior varia...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Neural Networks
سال: 2019
ISSN: 0893-6080
DOI: 10.1016/j.neunet.2019.05.003